DIONYSUS: Towards Query-aware Distributed Processing of RDF Graph Streams

Arguably, the most significant obstacle to handle the emerging application’s data deluge is to design a system that addresses the challenges for big data’s volume, velocity and variety. Work in RDF stream processing (RSP) systems partly addresses the challenge of variety by promoting the RDF model. However, challenges like volume, velocity are overlooked by existing approaches. These challenges demand optimised combination of scale-out and scaleup solutions. Furthermore, various other requirements for RSP systems, such as an efficient integration of distributed stream sources, storage of historical streams and their analysis, and integration of stateful operators to support complex event processing over streams are far from being addressed in an efficient way. Our vision is to design a general purpose RDF graph streaming system, which will be able to cope with distributed streams and shares local optimising strategies to allow different kinds of queries (analytical, streaming, sequence-based) through one query interface. The proposed system will offer a black-box solution that will allow analysts to tap in the goldmine of massive RDF graph streams. We consider the challenges and opportunities associated in designing such system, introduce our approaches to these topics, and discuss the components of our envisioned system.